Users are accessing different services such as a visual search service from a wide range of different computing devices (e.g., both mobile devices and non-mobile devices). For example, these different computing devices include home computers, work computers, mobile phones, mobile device, tablets, etc. Visual search has become a popular service. Users upload images to a server, which matches the images against other images stored in its database and eventually returns information about the uploaded images. The algorithms that match the query images against the database images are typically designed such that they can handle a certain amount of warping (i.e., translation, scale, rotation, and perspective effects). Also, these methods can detect objects in pictures that contain additional, irrelevant details (i.e., clutter). However, even though the algorithms are able to cope with these difficulties, the algorithms require more time and processing resources for coping with these difficulties. The irrelevant clutter in the camera image not only makes it harder for the server to find the object of interest, it also increases the size of the image that is sent to the server.